Extend huggingface with rerank #127297
Extend huggingface with rerank #127297
Conversation
elasticsearchmachine
added
needs:triage
| public List<InferenceServiceExtension.Factory> getInferenceServiceFactories() { | ||
| return List.of( | ||
| context -> new HuggingFaceElserService(httpFactory.get(), serviceComponents.get()), | ||
| context -> new HuggingFaceRerankService(httpFactory.get(), serviceComponents.get()), |
There was a problem hiding this comment.
Instead of creating a whole new service we'll want to add the functionality to the HuggingFaceService.
For example the OpenAI service supports many task types: https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/openai/OpenAiService.java#L175-L197
There was a problem hiding this comment.
Thank you, Jonathan. Now the HuggingFaceService is used
|
|
||
| import static org.elasticsearch.xpack.inference.common.Truncator.truncate; | ||
|
|
||
| public class HuggingFaceRequestRerankManager extends BaseRequestManager { |
There was a problem hiding this comment.
We're trying to move away from the request manager pattern because it adds duplicate code. Could you look into following the pattern we started here (we haven't refactored all the services yet but if it's possible to do for hugging face it'd be great if we could do it now)?
One option would be to leave the other hugging face request managers as they are (if possible, it may not be though) and then use one of the generic request managers like shown in the PR above for this new functionality.
We'll want to sync up with @Jan-Kazlouski-elastic to ensure that we're not duplicating that refactoring work though.
There was a problem hiding this comment.
We synced with Jan. I followed the changes he applied with GenericRequestManager
|
|
||
| @Override | ||
| public String modelId() { | ||
| return null; |
There was a problem hiding this comment.
I would be adding comments for such implementations.
There was a problem hiding this comment.
Thanks. Added the comment
|
Pinging @elastic/ml-core (Team:ML) |
elasticsearchmachine
removed
the
needs:triage
Evgenii-Kazannik
force-pushed
the
extend-huggingface-with-rerank
branch
from
586ae7f to
390b7e7
Compare
5 participants
gradle check?